Performance Measurement Under Increasing Environmental Uncertainty In The Context of Interval Type-2 Fuzzy Logic Based Robotic Sailing

Performance measurement of robotic controllers based on fuzzy logic, operating under uncertainty, is a subject area which has been somewhat ignored in the current literature. In this paper standard measures such as RMSE are shown to be inappropriate for use under conditions where the environmental uncertainty changes significantly between experiments. An overview of current methods which have been applied by other authors is presented, followed by a design of a more sophisticated method of comparison. This method is then applied to a robotic control problem to observe its outcome compared with a single measure. Results show that the technique described provides a more robust method of performance comparison than less complex methods allowing better comparisons to be drawn.Comment: International Conference on Fuzzy Systems 2013 (Fuzz-IEEE 2013

An investigation into the factors affecting performance of fuzzy logic systems

Fuzzy logic is a frequently used solution to control problems, especially when there are elements of human knowledge that may be incorporated into the system. Fuzzy logic comes in several varieties with the most common being based on either type-1 or type-2 fuzzy logic. Modifications to these standard varieties, termed Non-Stationary (NS) and Dual Surface (DS) are also investigated. Each variety allows a certain amount of flexibility in its expression. However, with this increased flexibility (and potentially performance) comes additional resource requirements: either during run time with higher processing and memory requirements; or at design time, with additional parameters requiring selection and optimisation. There have been several comparisons into the performance obtained from type-1 and type-2 investigating such factors as their internal configuration (such as membership functions as defined by their Footprint of Uncertainty), task difficulty and the environment in which the experiments are performed. However, no studies have been performed incorporating each of these factors with the goal of determining how they impact upon performance. The end goal of this work is the development of a methodology to understand which combination of conditions will cause type-2 control to consistently outperform type-1 based systems. This would enable the rationalisation of moving from a type-1 to a type-2 system, which is currently done without understanding if and how performance will increase with such a move. This thesis introduces a novel scheme by which several methods of comparing performance are employed to observe how the output and resulting performance levels change as factors including: controller configuration, task difficulty and environmental variability are varied. These methods are performed over three applications which gradually increase in complexity: a simple tipping example, a more developed simulation based on an autonomous sailing robots application and subsequent real-world experiments, which also involve the autonomous sailing problem. The first method of comparison studies how the rules which fire for a given input set change as the configuration of the fuzzy logic controller is increased. The second comparative technique investigates the control surfaces produced by a selection of fuzzy logic controllers to observe how they change as the internal configuration is changed. Observations such as the smoothing of the transitions between surfaces suggest that controllers with a larger FOU may give a better response. The third method for comparison is developed in which outputs from a controller operating in a simulated environment are compared to an ideal value, giving a single numeric output with which comparisons can be made. It was found that there are situations in which type-2 based fuzzy control outperforms type-1. However, these are found to be less common than expected. It is determined that this may be due to the simplicity of some of our case studies environments (especially the tipping example), where there may not be enough scope for large improvements to become apparent. These findings lay ground for future work in which (i) more developed and complex applications and (ii) a more tuned fuzzy system should be investigated to find if this will result in more obvious differences between configurations

An investigation into the factors affecting performance of fuzzy logic systems

Fuzzy logic is a frequently used solution to control problems, especially when there are elements of human knowledge that may be incorporated into the system. Fuzzy logic comes in several varieties with the most common being based on either type-1 or type-2 fuzzy logic. Modifications to these standard varieties, termed Non-Stationary (NS) and Dual Surface (DS) are also investigated. Each variety allows a certain amount of flexibility in its expression. However, with this increased flexibility (and potentially performance) comes additional resource requirements: either during run time with higher processing and memory requirements; or at design time, with additional parameters requiring selection and optimisation. There have been several comparisons into the performance obtained from type-1 and type-2 investigating such factors as their internal configuration (such as membership functions as defined by their Footprint of Uncertainty), task difficulty and the environment in which the experiments are performed. However, no studies have been performed incorporating each of these factors with the goal of determining how they impact upon performance. The end goal of this work is the development of a methodology to understand which combination of conditions will cause type-2 control to consistently outperform type-1 based systems. This would enable the rationalisation of moving from a type-1 to a type-2 system, which is currently done without understanding if and how performance will increase with such a move. This thesis introduces a novel scheme by which several methods of comparing performance are employed to observe how the output and resulting performance levels change as factors including: controller configuration, task difficulty and environmental variability are varied. These methods are performed over three applications which gradually increase in complexity: a simple tipping example, a more developed simulation based on an autonomous sailing robots application and subsequent real-world experiments, which also involve the autonomous sailing problem. The first method of comparison studies how the rules which fire for a given input set change as the configuration of the fuzzy logic controller is increased. The second comparative technique investigates the control surfaces produced by a selection of fuzzy logic controllers to observe how they change as the internal configuration is changed. Observations such as the smoothing of the transitions between surfaces suggest that controllers with a larger FOU may give a better response. The third method for comparison is developed in which outputs from a controller operating in a simulated environment are compared to an ideal value, giving a single numeric output with which comparisons can be made. It was found that there are situations in which type-2 based fuzzy control outperforms type-1. However, these are found to be less common than expected. It is determined that this may be due to the simplicity of some of our case studies environments (especially the tipping example), where there may not be enough scope for large improvements to become apparent. These findings lay ground for future work in which (i) more developed and complex applications and (ii) a more tuned fuzzy system should be investigated to find if this will result in more obvious differences between configurations

Performance measurement under increasing environmental uncertainty in the context of interval type-2 fuzzy logic based robotic sailing

Performance measurement of robotic controllers based on fuzzy logic, operating under uncertainty, is a subject area which has been somewhat ignored in the current literature. In this paper standard measures such as RMSE are shown to be inappropriate for use under conditions where the environmental uncertainty changes significantly between experiments. An overview of current methods which have been applied by other authors is presented, followed by a design of a more sophisticated method of comparison. This method is then applied to a robotic control problem to observe its outcome compared with a single measure. Results show that the technique described provides a more robust method of performance comparison than less complex methods allowing better comparisons to be drawn

A comparison of non-stationary, type-2 and dual surface fuzzy control

Type-1 fuzzy logic has frequently been used in control systems. However this method is sometimes shown to be too restrictive and unable to adapt in the presence of uncertainty. In this paper we compare type-1 fuzzy control with several other fuzzy approaches under a range of uncertain conditions. Interval type-2 and non-stationary fuzzy controllers are compared, along with ‘dual surface’ type-2 control, named due to utilising both the lower and upper values produced from standard interval type-2 systems. We tune a type-1 controller, then derive the membership functions and footprints of uncertainty from the type-1 system and evaluate them using a simulated autonomous sailing problem with varying amounts of environmental uncertainty. We show that while these more sophisticated controllers can produce better performance than the type-1 controller, this is not guaranteed and that selection of Footprint of Uncertainty (FOU) size has a large effect on this relative performance

An investigation into the relationship between type-2 FOU size and environmental uncertainty in robotic control

It has often been suggested that when faced with large amount of uncertainty in situations of automated control type-2 fuzzy logic based controllers will out perform the simpler type-1 varieties due to the latter lacking a mechanism to model this uncertainty and adapt accordingly. This paper aims to investigate this problem in detail and analyse when and the magnitude by which a type-2 controller will improve upon type-1 performance. A sailing robot is subjected to several experiments in which the uncertainty and the complexity of the sailing problem is gradually increased in order to observe the effects on measured performance. Improved performance is observed, however this does not occur in all cases. The size of the FOU is shown to be very important in the creation of the type-2 system with potentially severe performance penalties for incorrectly sized systems

A Comparison of Nonstationary , Type-2 and Dual Surface Fuzzy Control

Abstract—Type-1 fuzzy logic has frequently been used in control systems. However this method is sometimes shown to be too restrictive and unable to adapt in the presence of uncertainty. In this paper we compare type-1 fuzzy control with several other fuzzy approaches under a range of uncertain conditions. Interval type-2 and non-stationary fuzzy controllers are compared, along with ‘dual surface ’ type-2 control, named due to utilising both the lower and upper values produced from standard interval type-2 systems. We tune a type-1 controller, then derive the membership functions and footprints of uncertainty from the type-1 system and evaluate them using a simulated autonomous sailing problem with varying amounts of environmental uncertainty. We show that while these more sophisticated controllers can produce better performance than the type-1 controller, this is not guaranteed and that selection of Footprint of Uncertainty (FOU) size has a large effect on this relative performance